1 Problem: Anonymous Crowd Monitoring

Particularly in light of unusually warm weather 1, 2, 3, the risks of railway accidents has dramatically increased over the last years. Inspections using special trains are costly and infrequent (twice a week, 4 ), while changes can happen over the course of a single day.

2 Solution: Drone and Stationary Cameras, Real-time In-Memory Analysis

By mounting inexpensive portable imaging devices on each train, we can collect real-time image information on each of the tracks being crossed by processing the images using our Spark Image Layer.

2.1 How?

The first question is how the data can be processed. The basic work is done by a simple workflow on top of our Spark Image Layer. This abstracts away the complexities of cloud computing and distributed analysis. You focus only on the core task of image processing.

Beyond a single camera, our system scales linearly to multiple cameras and can distribute this computation across many computers to keep the computation real-time.

With cloud-integration and Big Data-based frameworks, even handling an entire city network with 100s of drones and cameras running continuously is an easy task without worrying about networks, topology, or fault-tolerance. Below is an example for 30 drones where the tasks are seamlessly, evenly divided among 50 different computers.

2.2 What?

The images which are collected by the drones at rate of 30 frames per second contain substantial dynamic information on not only persons, but vehicles, buildings, and landscapes. The first basic task is the segmentation of the people which can provide information on their number, movement, and behavior

  • Labels

The segmented image above can be transformed into quantitative metrics at each time point. These metrics can then be processed to extract relevant quality assessment information for the tracks.

The data can then be broken down into small scenes where the number and flow of people in each grouping can be evaluated.

Dynamic information can be obtained by tracking the person objects (below) from one scene to the next. As seen in the image below, these are simply outlines and completely un-identifiable back to the original person. This alleviates many privacy and personal information concerns.

  • Labels

The data can also be smoothed to show more clearly trends and person counts on a single image.

3 Technical Aspects

3.1 Streaming the Data

Once the cluster has been comissioned and you have the StreamingSparkContext called ssc (automatically provided in Databricks Cloud or Zeppelin), the data can be loaded using the Spark Image Layer. Since we are using real-time analysis, we acquire the images from a streaming source

val trainCam1 = TrainCameraReceiver("sbb://train-3275")
val trainCam2 = TrainCameraReceiver("sbb://train-3187")
val metaImageStream = ssc.receiverStream(trainCam1 ++ trainCam2)

Although we execute the command on one machine, the analysis will be distributed over the entire set of cluster resources available to ssc. To further process the images, we can take advantage of the rich set of functionality built into Spark Image Layer

def identifyTracks(time: Double, pos: GeoPos, inImage: Img[Byte]) = {
  // Run the image processing steps on all images
  val rawTrack = inImage.
    run("Median...","radius=3"). // filter out the noise
    run("Tubeness..."). // highlight tube-like structures (tracks)
    run("Threshold","OTSU") // threshold bright structures
  val trackShape = rawTrack.
    componentLabel(). // identify the components of the track
    filter(_.area>50). // take only the larger sized
    shapeAnalysis() // analyze the position and shape
  // return smoothness and separation based on the segmented image
  TrackInformation(smoothness=rawTrack.smoothness(),
                   separation=calcSep(trackShape))
}
// apply the operation to all images as they come in
val trackStream = metaImageStream.map(identifyTracks)

The detailed information with time, position and track data can then be compared to reference values to find changes and alert the user

trackStream.foreachRDD{
  trackStream =>
    val changePoints = trackStream.
      join(trackStandards).
      filter{
        case(newData,oldResults) => abs(oldResults-newData)>0.05
      }.collect()
      // update the map
      updateMap(changePoints)
      // send an alert
      changePoints.map(
        (newPt,_) => 
          sendAlert(newPt.pos+" at "+newPt.time+" has changed by more then 5%")
      )
}

The entire pipeline can then be started to run in real-time on all the new images as they stream in. If the tasks become more computationally intensive, then the computing power can be scaled up and down elastically.

4 Acknowledgements

Analysis powered by Spark Image Layer from 4Quant, Visualizations, Document Generation, and Maps provided by:

To cite ggplot2 in publications, please use:

H. Wickham. ggplot2: elegant graphics for data analysis. Springer New York, 2009.

A BibTeX entry for LaTeX users is

@Book{, author = {Hadley Wickham}, title = {ggplot2: elegant graphics for data analysis}, publisher = {Springer New York}, year = {2009}, isbn = {978-0-387-98140-6}, url = {http://had.co.nz/ggplot2/book}, }

To cite package ‘leaflet’ in publications use:

Joe Cheng and Yihui Xie (2014). leaflet: Create Interactive Web Maps with the JavaScript LeafLet Library. R package version 0.0.11. https://github.com/rstudio/leaflet

A BibTeX entry for LaTeX users is

@Manual{, title = {leaflet: Create Interactive Web Maps with the JavaScript LeafLet Library}, author = {Joe Cheng and Yihui Xie}, year = {2014}, note = {R package version 0.0.11}, url = {https://github.com/rstudio/leaflet}, }

To cite plyr in publications use:

Hadley Wickham (2011). The Split-Apply-Combine Strategy for Data Analysis. Journal of Statistical Software, 40(1), 1-29. URL http://www.jstatsoft.org/v40/i01/.

A BibTeX entry for LaTeX users is

@Article{, title = {The Split-Apply-Combine Strategy for Data Analysis}, author = {Hadley Wickham}, journal = {Journal of Statistical Software}, year = {2011}, volume = {40}, number = {1}, pages = {1–29}, url = {http://www.jstatsoft.org/v40/i01/}, }

To cite the ‘knitr’ package in publications use:

Yihui Xie (2015). knitr: A General-Purpose Package for Dynamic Report Generation in R. R package version 1.10.

Yihui Xie (2013) Dynamic Documents with R and knitr. Chapman and Hall/CRC. ISBN 978-1482203530

Yihui Xie (2014) knitr: A Comprehensive Tool for Reproducible Research in R. In Victoria Stodden, Friedrich Leisch and Roger D. Peng, editors, Implementing Reproducible Computational Research. Chapman and Hall/CRC. ISBN 978-1466561595

To cite package ‘rmarkdown’ in publications use:

JJ Allaire, Joe Cheng, Yihui Xie, Jonathan McPherson, Winston Chang, Jeff Allen, Hadley Wickham and Rob Hyndman (2015). rmarkdown: Dynamic Documents for R. R package version 0.7. http://CRAN.R-project.org/package=rmarkdown

A BibTeX entry for LaTeX users is

@Manual{, title = {rmarkdown: Dynamic Documents for R}, author = {JJ Allaire and Joe Cheng and Yihui Xie and Jonathan McPherson and Winston Chang and Jeff Allen and Hadley Wickham and Rob Hyndman}, year = {2015}, note = {R package version 0.7}, url = {http://CRAN.R-project.org/package=rmarkdown}, }

To cite package ‘DiagrammeR’ in publications use:

Knut Sveidqvist, Mike Bostock, Chris Pettitt, Mike Daines, Andrei Kashcha and Richard Iannone (2015). DiagrammeR: Create Graph Diagrams and Flowcharts Using R. R package version 0.7.

A BibTeX entry for LaTeX users is

@Manual{, title = {DiagrammeR: Create Graph Diagrams and Flowcharts Using R}, author = {Knut Sveidqvist and Mike Bostock and Chris Pettitt and Mike Daines and Andrei Kashcha and Richard Iannone}, year = {2015}, note = {R package version 0.7}, }